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ABSTRACT 



Social desirability responding (SDR) on surveys administered 
on the World Wide Web and on paper was examined, with 178 graduate and 
undergraduate students as participants. To assess the extent to which 
participants would demonstrate SDR, this study used the Balanced Inventory of 
Desirable Responding (BIDR) (Paulhus, 1993) . The BIDR consists of 40 items 
states as propositions. Respondents rate their agreement with each statement 
on a seven-point scale. Using a true experimental design, the study examined 
the impact of two independent variables, the participants' identif iability 
level (anonymous and non- anonymous) and the survey's administration mode 
(World Wide Web-administered and paper-administered) , on one dependent 
variable, the participants' social desirability response levels measured by 
the BIDR. Findings reveal no differences in SDR between the Web and the 
paper-administered survey conditions, and no differences in SDR between the 
anonymous and non-anonymous conditions. These findings and potential 
explanations are examined for consideration by anyone interested in using the 
Web to obtain accurate information from survey participants. (Contains 35 
references.) (AEF) 
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Abstract 



Social desirability > responding (SDR) on surveys administered on the World Wide Web (WWW) and on 
paper was examined using 178 graduate and undergraduate students randomly assigned to a 2 (survey's 
administration mode: WWW and paper) x 2 (participants' identifiability level: anonymous and non- anonymous) true 
experimental design. The findings reveal no differences in SDR between the WWW and the paper-administered 
survey conditions, and no differences in SDR between the anonymous and non-anonymous conditions. These 
findings and potential explanations are examined for consideration by anyone interested in using the WWW to 
obtain accurate information from survey participants . 

Introduction 

Response bias (i.e., the systematic tendency to respond to surveys, questionnaires, standardized tests, and 
other self-report measures on some basis other than the specific item content) continues to confound research 
findings. People’s reports of their own traits, attitudes, and behaviors often involve systematic bias that obscures 
measurement of content variables (Calsyn, 1999; Paulhus, 1991). For example, early research suggested that 
standard self-report methodologies distorted the reporting of racist attitudes (Sigall & Page, 1971), abnormal sexual 
attitudes (Knudson, Pope, & Irish, 1967), desirable behaviors (Phillips & Clancey, 1972), deviant behaviors (Clark 
& Tifft, 1966), and abortion (Wiseman, 1972). More recent studies have revealed a tendency among individuals to 
conceal truth when reporting unverifiable information (Lautenschlager & Flaherty, 1990), seeking employment 
(Calsyn & Klinkenberg, 1995), reporting information designed to impress others (Rosenfeld, Giacalone, & Riordan, 
1995), and when a respondent's anonymity is violated (Sproull & Kiesler, 1991). 

Among the most common forms of response bias reported in the literature are deviant responding (Berg, 
1967), careless responding (Meehl & Hathaway, 1946), consistent responding (Dillehay & Jemigan, 1970), item 
omission (Cronbach, 1946), acquiescence (Ray, 1983), and extremity bias (Hamilton, 1968; Peabody, 1962). 
However, perhaps the most frequently studied response bias is social desirability responding (SDR) (i.e., the 
tendency to provide answers which cause the respondent to look good) (Rosenfeld, Booth-Kewley, Edwards, & 
Thomas, 1996). As early as the 1930s, Bemreuter (1933) reported that psychometricians had already noted the 
problem of SDR effects on the validity of questionnaires. Years later, Meehl and Hathaway (1946) were able to cite 
eight measures specifically designed to index SDR in self-report measures. Since that time, SDR has been a major 
concern in measuring personality, psychopathology, attitudes, and self-reports of various forms of sensitive behavior 
(Paulhus, 1991). 

Recently, the proliferation of web-based and other computer-assisted means of acquiring information from 
individuals has raised concerns regarding how responses obtained through computers compare with responses 
obtained on paper instruments. Several published studies (see Booth-Kewley, Rosenfeld, & Edwards, 1993; 
Moorman & Podsakoff, 1992. for reviews) have reported that computer responses are more candid, less biased, and 
less influenced by social desirability than responses given on paper. However, very few studies have examined 
people’s responses on the World Wide Web (WWW). Furthermore, several research efforts have failed to replicate 
the findings of previous studies regarding SDR. For example, studies in the 1980s investigating the feasibility of a 
U S. Navy computer-based survey system failed to demonstrate that computer-administered surveys were superior at 
reducing SDR (Doherty & Thomas, 1986; Rosenfeld, Doherty, Vicino, Kantor, & Greaves, 1989; Vicino, 1989). In 
addition, Lautenschlager and Flaherty (1990) found that undergraduates responding to a computerized survey had 
higher scores on a self-deception scale than did their counterparts who used a paper survey. Extending this research, 
Booth-Kewley et al. (1993) had subjects complete surveys using computer or paper administration modes under 
anonymous or identified conditions. Consistent with Lautenschlager and Flaherty's (1990) results, Booth-Kewley 
and her colleagues found that identified respondents gave more socially desirable responses than did their 
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anonymous counterparts. However, Booth-Kewley et al. (1993) failed to replicate Lautenschlager and Flaherty's 
(1 990) finding of greater levels of SDR among participants who completed the survey by computer. 

The lack of consistent findings regarding the extent to which people demonstrate SDR on computer versus 
paper-administered instruments, combined with the scarcity of research in this area on people who respond to 
surveys and questionnaires administered through the WWW, was the purpose for conducting the current study. 
Based on previous research suggesting that computer-administered surveys yield more candid responses than do 
paper surveys, this study hypothesized that adult students taking a survey on the WWW would demonstrate 
significantly less SDR than would students taking the same survey on paper. Furthermore, based on previous 
research suggesting that participants would be more inclined to respond to survey items under conditions of 
anonymity, this study hypothesized that adult students taking the survey anonymously would demonstrate 
significantly less SDR than would respondents who were asked to identify themselves. 

The Study 

Participants 

178 undergraduate and graduate students at a large university in the southeastern United States, enrolled in 
introductory research and technology courses, participated in this study. 69% of the participants were female. The 
average age of participants was 34.2 years. 

Instrument 

To assess the extent to which participants would demonstrate SDR, this study used the Balanced Inventory 
of Desirable Responding (BIDR) (Paulhus, 1993). The BIDR consists of 40 items stated as propositions. 
Respondents rate their agreement with each statement on a seven-point scale. The scoring key is balanced. After 
reversing the negatively keyed items, one point is added for each extreme response (six or seven). This method of 
scoring ensures that high scores are attained only by subjects who give exaggeratedly desirable responses. The 40 
items are then summed to yield an individual's overall level of SDR. 

Several studies have established the reliability and validity of the BIDR. With respect to internal validity, 
values of coefficient alpha have ranged from .68 to .80 and from .75 to .86 (Mellor, Conroy, & Masteller, 1986; 
Paulhus, 1984, 1993; Quinn, 1989). Demonstrating concurrent validity as a measure of SDR, the sum of the 40 
items on the BIDR correlated .71 with the Marlowc-Crowne Social Desirability scale (Crowne & Marlowe, 1960) 
and .80 with the Multidimensional Social Desirability Inventory of Jacobson, Kellogg, Cauce, and Slavin (1977). 
Supporting the construct validity of the BIDR, Paulhus (1991) discovered that high deception subjects were more 
likely than lows to show a self-serving bias after a failure experience. High self-deception subjects also showed 
more illusion of control, belief that they were safe drivers, and proneness to love (Paulhus & Reid, 1991) and to 
intrinsic religiosity (Leak & Fish, 1989). 

Procedures 

Using a true experimental design, this study examined the impact of two independent variables — the 
participants' identifiability level (i.e., anonymous and non-anonymous) and the survey's administration mode (i.e., 
WWW-administered and paper-administered) — on one dependent variable — the participants’ social desirability 
response levels measured by the BIDR. Using a random number table, 283 potential participants were assigned 
randomly to one of four conditions: (a) anonymous/WWW-administered survey (n=75); (b) non- 

anonymous/WWW-administered survey (n=78); (c) anonymous/paper-administered survey (n=63); and (d) non- 
anonymous/paper-administered survey (n=67). 

Prior to their departure from the classroom during an initial session in the course, participants in all four 
conditions were provided a manila envelope by their professor, who was instructed to say, " Prior to our next class 
attendance, please follow the instructions contained in your envelope . Those instructions will require you to 
individually complete a vety short survey that we will use later in this course when we discuss data collection 
techniques. Remember, please follow the instructions contained in your envelope by our next class attendance. 
Thank you.” 

For the participants in the paper-administered survey groups, the manila envelopes contained a copy of the 
Balanced Inventory of Desirable Responding (BIDR). One-half of the BIDRs in those envelopes (i.e., the non- 
anonymous/paper-administered survey group) were preceded by the written instructions, ” Please complete the 
following survey and mail it in the self-addressed envelope prior to our next class attendance. The survey contains 
40 items and takes approximately 1 0 minutes to complete. Using the scale below as a guide, write a number in each 
blank to indicate your agreement with the statement. For the purpose of accountability, please be certain that you 
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print your name legibly in the blank provided': The results of this swvey will be completely confidential. The results 
will be reported onlv in the aggregate: therefore, at no time will you be identified individually. ' 

The other one-half of the BIDRs in thCrse envelopes (i.e., the anonymour/pper-administered survey group) 
were preceded by the same written instructions, witli one exception , “ jktc phrase, For the purpose of 
accountability, please be certain that you print your name legibly in the blank provided," was replaced with the 
phrase, "To maintain anonymity, please do not indicate your none cry where on the survey. 

For the participants in the WWW-adntinisiercd survey groups, one-half of the manila envelopes (i.e., the 
non-anonymous/WWW-administered survey group) contained a sheet of paper with the written instructions, "Please 
complete the survey that you will find on the World Wide Web at http://education.uncc.edu/surveyl.htm prior to the 
next class attendance. The results of this survey will be completely confidential. The results will be reported only in 
the aggregate : therefore, at no time will you be identified individually. For participants in this condition, the BIDR 
at the prescribed web site address included the following additional instructions, "This survey contains 40 items and 
takes approximately 10 minutes to complete. Using the scale below as a guide, type a number in each blank to 
indicate your agreement vsiilt the statement. For the purpose of accountability, please be certain that you type your 
name in the blank pro vidfd. " 

The other o.Yic-half of these manila envelopes (i.e., the anonymous/WWW-administered survey group) 
contained a,s!ieet of paper, with the. same written instructions, with two exceptions - the WWW address for 
accessing the survey for participants in this condition was changed from "education.uncc.edu/surveyI.htm " to 
"education.uncc.edu/ respond I him, " and the phrase, “ For the purpose of accountability, please be certain that you 
type your name in the blank provided. " was replaced with the phrase, "To maintain anonymity, please do not 
indicate jyour name anywhere on the survey. " 

Of the 283 potential participants in this study, 181 students completed a survey (i.e., a 64% response rate). 
However, three students' survey responses were excluded from the data analysis because these students incorrectly 
completed the survey. Of the remaining 178 surveys, 44 surveys were in the anonymousAVorld Wide Web- 
admi^istered condition (i.e., a 59% response rate), 50 surveys were in the non-anonymous/World Wide Web- 
adnyinistered condition (i.e., a 64% response rate), 44 surveys were in the anonymous/paper-administered condition 
(i.e.f, a 70% response rate). and_40 surveys were in the non-anonymous/papcr-administcrcd condition (i.e., a 60% 
respionse rate). 

A 2x2 analysis of variance (ANOVA) was conducted with the survey’s administration mode (i.e., WWW- 
ad/ministered and paper f adniinistercd). and the participants' identifiability level (i.e., anonymous and non- 
■'anonymous) as the independent variables and the participants’ SDR levels measured by the BIDR as the dependent 
variable. 

Findings 

Means, standard deviations, and sample sizes for the social desirability levels by condition are presented in 
Table 1. The results of the ANOVA arc presented in Table 2. The main effect for survey administration mode was 
not statistically significant (/^ 1,1 74)=. 071, />>.05). Students taking the survey on the WWW (M— 15.33, sd=6. 23) 
did not demonstrate significantly less SDR than did adult students taking the same survey on paper (A/= 15.07, 
5^=5.50). Furthermore, the main effect for participants' identifiability level was not statistically significant 
(F( 1,1 74)=. 150, p>. 05). Students taking the survey anonymously (A/* 5 1 5.03, $d=5Al) did not demonstrate 

significantly less SDR' than did survey-takers who were asked to identify themselves (A/=15.39, sd= 6.29). Finally, 
there was no statistically significant interaction (F( 1,174)= 027, p>.05), suggesting that no differential effect on 
SDR was noted with the combination of independent variables. 



Conclusions 

The hypotheses of this study were based on previous research; therefore, it is important to examine possible 
reasons for the non-statistically significant findings. One explanation may be that, unlike the current study, most 
previous studies linking lower anonymity with higher levels of SDR administered their surveys within the context of 
the experimental setting. For example, Lautenschlager and Flaherty (1990) asked college students to complete the 
BIDR in a small office on the site of the study. Similarly, Rosenfeld, Booth-Kewley, Edwards, and Thomas (1996) 
administered the BIDR to Navy recruits in a large testing room immediately after the sailors received instructions 
about the study. However, in the current study, after the professor distributed the manila folders, students were 
allowed to depart the classroom with the expectation that they would complete the survey on the WWW or paper 
prior to the next lesson.. Undoubtedly, students completed the survey in many different locations - at home, at 
work, at school, in a computer lab, and so forth. As a result, even those students in the non-anonymous WWW and 
paper-administered survey conditions whose instructions included a directive to type or print their names on the 
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survey may have felt a sense of anonymity as they completed the BIDR. This pervasive sense of anonymity 
experienced by the participants may have mitigated the effects of self-identification created in the study, thereby 
contributing to the lack of a statistically significant main effect for participants' identifiability level. 

Another potential contributor to this study's findings may have been the written instructions provided to the 
participants. In all experimental conditions, the written instructions included the sentences, The results of this 
survey will be completely confidential The results will be reported only in the aggregate; therefore, at no time will 
you be identified individually. " Although these sentences were inserted to ensure compliance with the 1974 Buckley 
Amendment, their inclusion may have lessened the extent to which participants in the non-anonymous conditions 
perceived that they could be identified. In effect, these sentences may have caused all participants to believe that 
their identity was completely protected, thereby mollifying the impact of anonymity on participants demonstration 

of SDR. . 

Furthermore, research has suggested that a survey-taker's perception of the verifiability of her or his survey 
responses may impact the extent to which the survey-taker stretches the truth in an effort to make a good impression 
(Lautenschlager & Flaherty, 1990). Specifically, when respondents believe that their answers cannot be validated, 
they tend to exhibit higher levels of SDR than when they think that their responses are verifiable. In the current 
study, however, participants were told that the results of the survey would be used in an upcoming discussion of data 
collection techniques. As a result, participants may have believed that the survey responses were being verified, 
thereby negating any SDR effects prompted by the method of survey administration (i.e., WWW or paper). 

In the past, one factor often associated with lower levels of SDR in computer-administered survey 
responses than in paper-administered survey responses has been the standardization that computer administration 
affords (Feuer, 1986). In computer-administered surveys, SDR may be reduced by controlling the respondent's 
ability to preview, skip, review items, and change responses. In other words, the greater structure imposed by the 
computer mode of survey administration may limit respondents’ ability to reveal themselves in the best possible 
light. However, in the current study, the survey's presentation on the WWW was designed to maximize participants' 
freedom to negotiate the instrument. Students who accessed the BIDR through the WWW had complete latitude to 
preview, skip, change, and review their responses to the items prior to submitting their surveys electronically. As a 
result, the restrictions often inherent in computer survey administration which lead to lower levels of SDR were not 
evident in this study, perhaps contributing to the non-statistically significant main effect for survey administration 
mode. 

Finally, although early research revealed that computer-administered survey settings seemed to reduce SDR 
because those settings offered greater anonymity and were perceived as impersonal and nonjudgmental, recent 
studies have discovered a growing concern among many survey-takers that computers are becoming overly intrusive 
(Rosenfeld, Booth-Kewley, Edwards, & Thomas, 1996). This concern, sometimes labeled the 'big brother 
syndrome" (Martin & Nagao, 1989), suggests that people are becoming more aware that computer communications 
can be monitored and shared. Computer-users who suspect "big brother monitoring" have reported increased 
anxiety, fatigue, stress, and reduced job satisfaction (Eisman, 1991; Iadipaolo, 1992). In the current study, students 
in the computer-administered survey condition may have felt that their responses, even in the anonymous condition, 
could and perhaps would be traced to them through the WWW. As a result, participants may have felt less inclined 
to present themselves in a truthful manner. 

The growing popularity of computers throughout much of the world suggests that computer administration 
of surveys will continue to increase in the future. Therefore, it is important to know how survey responses obtained 
through computers compare with responses obtained on paper instruments. Although some research has reported 
that computer responses are more candid, less biased, and less influenced by social desirability than responses 
provided on paper, the current study using the WWW as the means by which to administer computer-based surveys 
did not support these findings. Students taking a survey on the WWW did not demonstrate significantly less SDR 
than did students taking the same survey on paper, and students taking a survey anonymously did not demonstrate 
significantly less SDR than did survey-takers who are asked to identify themselves. Unlike traditional computer- 
based modes of survey administration in which people demonstrate less SDR than people taking a survey by paper, 
this study suggests that people who take surveys on the WWW demonstrate SDR at levels comparable to those who 
take paper surveys. Although increased objectivity and cost effectiveness have often been associated with computer 
administration of surveys relative to paper administration of surveys, this study’s findings should be considered 
carefully by all professions interested in using the WWW to obtain truthful and accurate information from survey- 
takers. 
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Table 1 



Means, Standard Deviations, and Sample Sizes of Social Desirability Responding level by Participants' 
Identifiability Level and Survey's Administration Mode 



Administration Mode 



World Wide Web- Paper- 

Administered Administered Overall 





M 


SD 


n 


M 


SD 


n 


M 


SD 


n 


Participants' Identifiability 
















Anonymous 


15.07 


5.73 


44 


14.98 


5.25 


44 


15.03 


5.47 


88 


Non-anonymous 


15.56 


6.69 


50 


15.18 


5.83 


40 


15.39 


6.29 


90 


Overall 


15.33 


6.23 


94 


15.07 


5.50 


84 


15.21 


5.88 


178 



Table 2 

2x2 A NOVA of Effects of Participants' Identifiability Level and Survey's Administration Mode on Social Desirability 
Responding Level 





SS 


df 


MS 


F 


Sig 


Participants' Identifiability Level (PIL) 


5.256 


1 


5.256 


.150 


.699 


Survey's Administration Mode (SAM) 


2.504 


1 


2.504 


.071 


.790 


PIL x SAM 


.956 


1 


.956 


.027 


.869 


Residual 


6115.868 


174 


35.149 







Note: £>.05 
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